Upgrade to Python 3.11 and fix most existing vulnerabilities #7510
Conversation
* RestrictedPython to 8.0, authlib to latest * Ensure werkzeug is latest 2.3.x
* Python 3.11 upgrade * Updates to protobuf, pyathena, pydgraph * Upgrade snowflake-connector-python
Dockerfile
Outdated
| # Controls whether to instrument code for coverage information | ||
| ARG code_coverage | ||
| ENV BABEL_ENV=${code_coverage:+test} | ||
| ENV GITHUB_PAT=${GITHUB_PAT} |
There was a problem hiding this comment.
This can probably now be removed, I don't think it's required.
| RUN curl -sSL https://install.python-poetry.org | python3 - | ||
|
|
||
| # Avoid crashes, including corrupted cache artifacts, when building multi-platform images with GitHub Actions. | ||
| RUN /etc/poetry/bin/poetry cache clear pypi --all |
| - .env | ||
| x-redash-environment: &redash-environment | ||
| REDASH_HOST: http://localhost:5001 | ||
| REDASH_HOST: http://localhost:5000 |
There was a problem hiding this comment.
Is 5001 now the default? Restore this.
| requests-aws-sign = "0.1.5" | ||
| sasl = ">=0.1.3" | ||
| # TODO: https://github.com/cloudera/python-sasl/pull/31 - outstanding PR to be merged | ||
| sasl = {git = "https://github.com/gughy8/python-sasl.git#master"} |
There was a problem hiding this comment.
Not that this is a forked dependency, which is not ideal.
There was a problem hiding this comment.
There was a separate PR merged with version 0.4a1 which might replace this (utilising cythonize) - https://github.com/cloudera/python-sasl/commits/master/
Dockerfile
Outdated
| ARG install_groups="main,all_ds,dev" | ||
| RUN /etc/poetry/bin/poetry install --only $install_groups $POETRY_OPTIONS | ||
| RUN --mount=type=cache,target=/root/.cache/pypoetry \ | ||
| /etc/poetry/bin/poetry install --only $install_groups $POETRY_OPTIONS |
There was a problem hiding this comment.
Repeat builds with poetry install take 10 minutes on Mac M1 2021 Macbook Pro! I got it down to 9-10 seconds through Docker BuildKit caching.
There was a problem hiding this comment.
Repeat builds with poetry install take 10 minutes on Mac M1 2021 Macbook Pro! I got it down to 9-10 seconds through Docker BuildKit caching.
Very interesting! This might deserve it's own PR
|
We need to hold See also #7482 (reply in thread) |
| werkzeug = "2.3.8" | ||
| wtforms = "2.2.1" | ||
| xlsxwriter = "1.2.2" | ||
| xmlschema = "2.5.1" |
There was a problem hiding this comment.
Required for SAML compatibility (legacy schema validation)
wtfiwtz
changed the title
|
There is a potential issue with the 
|
What type of PR is this?
Description
Upgrade to Python 3.11, which allows some other version upgrades of vulnerable libraries.
How is this tested?
Spinning up in Fargate and testing by hand (predominantly Athena in AWS as the data source)
Related Tickets & Documents
orchestrated-io#12
orchestrated-io#13
orchestrated-io#15
orchestrated-io#16
cloudera/python-sasl#30
cloudera/python-sasl#31
cloudera/python-sasl#32
Mobile & Desktop Screenshots/Recordings (if there are UI changes)
With
trixie-proposed-updatesenabled